Founding developers

BBA & NT

Long-term collaborators

\(+\) neurodebian, slicer, brainsfit, nipype, itk and more …

a pride: common way of doing things

… in a competitive world …

Definitions

• Registration \(=\) estimate an “optimal” geometric mapping between image pairs or image sets (e.g. Affine)

• Similarity \(=\) a function relating one image to another, given a transformation (e.g. mutual information)

• Diffeomorphisms \(=\) differentiable map with differentiable inverse (e.g. “silly putty”, viscous fluid)

• Segmentation \(=\) labeling tissue or anatomy in images, usually automated (e.g. K-means)

• Multivariate \(=\) using many voxels or measurements at once (e.g. PCA, \(p >> n\) ridge regression)

• Multiple modality \(=\) using many modalities at once (e.g. DTI and T1 and BOLD)

• MALF: multi-atlas label fusion - using anatomical dictionaries to label new data

• Solutions to challenging statistical image processing problems usually need elements from each of the above

Image mapping & perception: 1878

• Francis Galton: Can we see criminality in the face?

• (maybe he should have used ANTs?)

Image mapping & biology: 1917

D’Arcy Thompson

Initial scope

… just do a better registration (tell story) …

ANTs Lineage

References: Horn and Schunck (1981), Gee, Reivich, and Bajcsy (1993), Grenander (1993), Thompson et al. (2001), Miller, Trouve, and Younes (2002), Shen and Davatzikos (2002), Arnold (2014), Thirion (1998), Rueckert et al. (1999), Fischl (2012), Ashburner (2012)

Diffeomorphisms

plausible physical modeling of large, invertible deformations

“differentiable map with differentiable inverse”

Fine-grained and flexible maps

… to correct a misconception about diffeomorphisms …

Diffeomorphisms: image parameterization in a metric space

General purpose library for multivariate image registration, segmentation & statistical analysis tools

• 170,000+ lines of C++, 6\(+\) years of work, 15+ collaborators.

• Generic mathematical methods that are tunable for application specific domains: no-free lunch

• Deep testing on multiple platforms … osx, linux, windows.

• Several “wins” in public knock-abouts ( Klein 2009, Murphy 2011, SATA 2012 and 2013, BRATS 2013, others )

    An algorithm must use prior knowledge about a problem
    to do well on that problem

ANTs: Beyond Registration

Atropos segmentation, N4 inhomogeneity correction, Eigenanatomy, SCCAN, Prior-constrained PCA, and atlas-based label fusion and MALF (powerful expert systems for segmentation)

On documentation

documentation is important

On documentation

… developers can be blind to doc deficiencies

while users are blind to what we provide! # ANTs Impact

ANTs Statistics

ANTs & Neuroscience

We need statistical image analysis
at several scales in modern neuroscience

• Macro: in vivo structural and functional MRI

• Micro: high-resolution post-mortem MRI links with in vivo MRI

• Nano: neuron reconstruction …

• Solutions that are consistent across these scales have the potential to build multi-scale feature sets or templates and provide new insights into brain structure and function

• E.g. Parcellation constraints based on histology, tractography, function …

• Statistical definitions of anatomy/pathology?

• Reinvention of these solutions within each lab … can we mitigate this?

• Reduce, reuse, recycle …

Financial impact

• The high error rate in software makes testing for quality assurance critically important.

• Bugs exist Everywhere: so don’t freak out if we admit we have bugs: we’re just being honest ( and not all devs are )

• Virtually every business in the United States now depends on software for development, production, distribution, and after-sales support of products and services.

• A 2002 NIST study estimated the direct costs to the software supply chain due to failure to identify (successfully test for) “bugs”

• We are better at this than most … not perfect

• The estimate of direct costs compiled from industry survey data for the U.S. economy was $60 billion per year

• … this estimate did not include costs to end users such as lost business (for example, the cost of shutting down the New York Mercantile Exchange in 1998 due to a software failure).

Reproducibility

• Consider the recent paper: Freesurfer, laplacian, DiReCT thickness in longitudinal stroke

• We “pre-wrote” a comment: Instrumentation bias in the use and evaluation of scientific software

• With these guys:

• 

Hired by Google

“Optimal” templates (?)

• unbiased wrt measurement space

• space is non-linear: reference matters

• encodes prior information: still just averages (usually)

• concept extends across modalities, anatomy, temporality, etc

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

Faces, brains, whatever …

see ANTs site and ANTs Documentation page

Templates from ANTs Colleagues ( not us )

Templates unlock modalities

Cortical thickness studies:

ANTs-related background

• Sandy, Brian, and DiReCT
• used other tools such as FMRIB’s Automated Segmentation Tool (FAST)
• KellySlater.cxx
• scripts “adequate for Penn data”
• KellyKapowski.cxx and antsCorticalThickness.sh
• “Let’s evaluate on open data.”
• and, eventually, “Let’s compare with FreeSurfer.”

The ANTs structural brain mapping workflow

Basic components of the pipeline

1. template building (offline)
2. brain extraction
3. cortical thickness estimation
4. cortical parcellation

Template building

Tailor data to your specific cohort

• Templates representing the average mean shape and intensity are built directly from the cohort to be analyzed, e.g. pediatric vs. middle-aged brains.
• Acquisition and anonymization (e.g. defacing) protocols are often different.

Template building (cont.)

Each template is processed to produce auxiliary images which are used for brain extraction and brain segmentation.

Brain extraction comparison
ANTs vs. FreeSurfer

Comparison with de facto standard FreeSurfer package. Note the difference in separation of the gray matter from the surrounding CSF. (0 failures out of 1205 scans)

Brain segmentation

Randomly selected healthy individuals. Atropos gets good performance across ages.

Cortical thickness estimation

In contrast to FreeSurfer which warps coupled surface meshes to segment the gray matter, ANTs diffeomorphically registers the white matter to the combined gray/white matters while simultaneously estimating thickness.

But without ground truth, how does one evaluate the pipeline?

• Use age and gender (demographic information) in training/prediction paradigm
• Public data sets (IXI, NKI, Oasis, MMRR)
• \(>\) 1200 subjects, age 7 to over 90 years old

Prediction from cortical thickness data:
gender and age

Regional importance
comparison

\(AGE \sim VOLUME + GENDER + \sum_{i=1}^{62} T(DKT_i)\)

ANTs (left) vs. FreeSurfer (right)

Regional importance comparison (cont.)

Cortical thickness extensions

antsLongitudinalCorticalThickness.sh

Longitudinal TBI with lesions (segmentation)

Longitudinal TBI with lesions (cortical thickness)

antsCorticalThickness.sh is adaptable!

1. Register subject (or single subject template) to normal template.

2. Transform lesion mask to normal template.

3. Create additional “lesion” prior, i.e. SmoothImage 3 ${lesionMask} 1 ${lesionPrior} 1.

4. Subtract out lesion prior from all other priors and keep values \(\in [0,1]\).

Modified template spatial priors

antsCorticalThickness.sh using lesion prior

Only change to the command call is an additional ‘-c WM[7]’ which means “combine the 7^th prior, i.e. lesion, to the white matter for cortical thickness calculation.”

multivariate statistical fields arise from fused modalities

Many opportunities for statistical advancements

ITK+ANTs+R = ANTsR

Agnostic statistics

McMillan et al.

A Quick ANTsR example

This is an executable ANTsR code block - N-dimensional statistics to go with our N-dimensional image processing software!

library(ANTsR)
dim<-2
filename<-getANTsRData('r16')
img<-antsImageRead( filename , dim )
filename<-getANTsRData('r64')
img2<-antsImageRead( filename , dim )
mask<-getMask(img,50,max(img),T)
mask2<-getMask(img,150,max(img),T)
nvox<-sum( mask == 1 )
nvox2<-sum( mask2 == 1 )

The brain has 18026 voxels …

A Quick ANTsR example

Simulate a population morphometry study - a “VBM” …

simnum<-10
imglist<-list()
imglist2<-list()
for ( i in 1:simnum ) {
  img1sim<-antsImageClone(img)
  img1sim[ mask==1 ]<-rnorm(nvox,mean=0.5)
  img1sim[ mask2==1 ]<-rnorm(nvox2,mean=2.0)
  img2sim<-antsImageClone(img2)
  img2sim[ mask==1 ]<-rnorm(nvox,mean=0.20)
  imglist<-lappend(imglist,img1sim)
  imglist2<-lappend(imglist2,img2sim)
}
imglist<-lappend( imglist, imglist2 )
mat<-imageListToMatrix( imglist, mask )
DX<-factor( c( rep(0,simnum), rep(1,simnum) ) )
mylmresults<-bigLMStats( lm( mat ~ DX ) )
qvals<-p.adjust( mylmresults$pval.model )

The minimum q-value is 4.491182310^{-6} …

Visualize the histograms of effects

whichvox<-qvals < 1.e-2
voxdf<-data.frame( volume=c( as.numeric( mat[,whichvox] ) ), DX=DX )
ggplot(voxdf, aes(volume, fill = DX)) + geom_density(alpha = 0.2)

Visualize the anatomical distribution

plot.antsImage(img,functional=list(betas),threshold=thresh,
  outname=ofn)

Network visualization

see ?plotBasicNetwork

The power of ANTs \(+\) R \(\rightarrow\)
Reproducible imaging science

… used in “Sparse canonical correlation analysis relates network-level atrophy to multivariate cognitive measures in a neurodegenerative population” and several upcoming …

Questions driving ANTs (\(+\) Refs.1)

• how should we geometrically transform anatomical coordinates?

  • syn paper - our geometric transformation model of choice
  • recent B-spline alternative/improvement

• how should we measure pairwise image similarity?

  • similarity metric evaluation compares functions for computing rigid or affine transformations between images

• what if this pair has rgb/vector/tensor voxels?

  • dti similarity

• how do we extend from pairs to hundreds or thousands of pairs of images?

  • optimal templates and spatial priors
  • species specific templates/priors in chimps and canines
  • ants “big data”

• how do we fuse multiple modality images at the subject and population levels?

  • ants auxiliary modality study

Questions driving ANTs (\(+\) Refs.2)

• can diffeomorphisms improve cortical thickness measurement?

• how might we efficiently cluster the statistical fields that arise in image analysis?

  • Atropos segmentation and N4 inhomogeneity correction
  • Eigenanatomy for sparse imaging-specific PCA

• how to cluster such fields when we have supervision?

  • sparse canonical correlation analysis for neuroimaging
  • Prior-constrained PCA
  • atlas-based label fusion and MALF - powerful expert systems for segmentation

• how do we implement a fully multivariate interpretable brain and behavior study?

  • SCCAN for imaging & cognition

• how do we extend these ideas to functional MRI & decoding?

  • recent unpublished software
  • recent work with Ben Kandel

ANTs longitudinal analysis

• Longitudinal image processing issues
  • registration induced bias
  • general & TBI-specific issues in longitudinal analysis
  • reproducibility of CBF
  • structure-specific analysis
• An early study of longitudinal cortical change in ALS
  • ALS atrophy rates
• Extension of standard-setting ants cortical thickness pipeline to longitudinal data
  • DynANTs (unpublished)

Challenges: Computational and Scientific

• Scalability: need to fuse feature selection methods with transformation optimization
• Scalability: need to leverage existing ITK streaming infrastructure in application level tool
• Domain expertise: Customizable for specific problems but sometimes not specific enough
• Rapid development: colleagues still need familiarity with compilation for latest ANTs features
• Latest theoretical advances in registration not yet wrapped for users
• Need more Documentation & testing …

Recap

• Powerful, general-purpose, well-evaluated registration and segmentation.

• Differentiable maps with differentiable inverse \(+\) statistics in these spaces

• Evaluated in multiple problem domains via internal studies & open competition

• Borg philosophy: “best of” from I/O, to processing to statistical methods

• Open source, testing, many examples, consistent style, multiple platforms, active community support …

• Integration with R \(+\) novel tools for prediction, decoding, high-to-low dimensional statistics.

• Collaborations with neurodebian, slicer, brainsfit, nipype, itk and more …

Tools you can use for imaging science

• Core developers: B. Avants, N. Tustison, H. J. Johnson, J. T. Duda

• Many contributors, including users …

• Multi-platform, multi-threaded C++ stnava.github.io/ANTs

• Developed in conjunction with http://www.itk.org/

• R wrapping and extension stnava.github.io/ANTsR

• rapid development, regular testing \(+\) many eyes \(\rightarrow\) bugs are shallow